Display control device

ABSTRACT

A display control device includes: an image data acquisition unit configured to acquire image data as a result of imaging by an imaging unit that images a situation around a vehicle; and a display processing unit configured to display, on a display unit, a peripheral image indicating the situation around the vehicle generated based on the image data and also display, on the peripheral image, a first vehicle image indicating a current state of the vehicle and a second vehicle image indicating a future state of the vehicle when the vehicle moves toward a target position and to change a display mode of at least one of the first vehicle image and the second vehicle image according to a remaining distance to the target position of the vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 U.S.C. § 119to Japanese Patent Application 2019-011693, filed on Jan. 25, 2019, theentire contents of which are incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates to a display control device.

BACKGROUND DISCUSSION

In the related art, various techniques have been studied to visuallynotify a vehicle occupant of a situation around a vehicle using imagedata as a result of imaging by an imaging unit that images the situationaround the vehicle. See, e.g., JP 3300334B (Reference 1), JP2017-175182A (Reference 2), JP 3947375B (Reference 3), and JP 5522492B(Reference 4).

In the conventional techniques as described above, for example, when thevehicle moves toward a target position, it is necessary to notify theoccupant of the mode of movement of the vehicle to the target positionalong with the situation around the vehicle in an easy-to-understandmanner.

Thus, a need exists for a display control device which is notsusceptible to the drawback mentioned above.

SUMMARY

A display control device as an aspect of this disclosure includes animage data acquisition unit configured to acquire image data as a resultof imaging by an imaging unit that images a situation around a vehicleand a display processing unit configured to display, on a display unit,a peripheral image indicating the situation around the vehicle generatedbased on the image data and also display, on the peripheral image, afirst vehicle image indicating a current state of the vehicle and asecond vehicle image indicating a future state of the vehicle when thevehicle moves toward a target position and to change a display mode ofat least one of the first vehicle image and the second vehicle imageaccording to a remaining distance to the target position of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and additional features and characteristics of thisdisclosure will become more apparent from the following detaileddescription considered with the reference to the accompanying drawings,wherein:

FIG. 1 is an exemplary and schematic diagram illustrating aconfiguration in a vehicle room of a vehicle according to an embodiment;

FIG. 2 is an exemplary and schematic diagram illustrating an appearanceof the vehicle according to the embodiment as viewed from above;

FIG. 3 is an exemplary and schematic block diagram illustrating a systemconfiguration of the vehicle according to the embodiment;

FIG. 4 is an exemplary and schematic block diagram illustratingfunctions of a display control device according to the embodiment;

FIG. 5 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image according to the embodiment;

FIG. 6 is an exemplary and schematic image diagram for explaining anexample of a relationship between the display position of a real vehicleimage and the display position of a virtual vehicle image in theembodiment;

FIG. 7 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image, corresponding to the example illustrated in FIG. 6, inthe embodiment;

FIG. 8 is an exemplary and schematic image diagram for explaining anexample of a relationship between the display position of a real vehicleimage and the display position of a virtual vehicle image, differentfrom FIG. 6, in the embodiment;

FIG. 9 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image, corresponding to the example illustrated in FIG. 8, inthe embodiment;

FIG. 10 is an exemplary and schematic diagram illustrating an example ofa display mode of a real vehicle image and a virtual vehicle image on aperipheral image according to the embodiment;

FIG. 11 is an exemplary and schematic diagram illustrating an example ofadjustment of the color (and/or brightness) of a virtual vehicle imagethat may be executed in the embodiment;

FIG. 12 is an exemplary and schematic diagram illustrating an example ofadjustment of the color (and/or brightness) of a real vehicle image thatmay be executed in the embodiment; and

FIG. 13 is an exemplary and schematic flowchart illustrating a series ofprocessings executed by the display control device according to theembodiment to display a real vehicle image and a virtual vehicle image.

DETAILED DESCRIPTION

Hereinafter, embodiments and modifications disclosed here will bedescribed with reference to the drawings. Configurations of theembodiments described later and actions, results, and effects providedby the configurations are given by way of example and are not limited tothe following description.

EMBODIMENT

First, a schematic configuration of a vehicle 1 according to anembodiment will be described with reference to FIGS. 1 and 2. FIG. 1 isan exemplary and schematic diagram illustrating a configuration in avehicle room 2 a of the vehicle 1 according to the embodiment, and FIG.2 is an exemplary and schematic diagram illustrating an appearance ofthe vehicle 1 according to the embodiment as viewed from above.

As illustrated in FIG. 1, the vehicle 1 according to the embodimentincludes the vehicle room 2 a in which an occupant including a driver asa user gets. In the vehicle room 2 a, a braking unit (braking operationunit) 301 a, an acceleration unit (acceleration operation unit) 302 a, asteering unit 303 a, a transmission unit (transmission operation unit)304 a, and the like are provided in a state where the user may operatethese units from a seat 2 b.

The braking unit 301 a is, for example, a brake pedal provided under thedriver's foot, and the acceleration unit 302 a is, for example, anaccelerator pedal provided under the driver's foot. Further, thesteering unit 303 a is, for example, a steering wheel protruding from adashboard (instrument panel), and the transmission unit 304 a is, forexample, a shift lever protruding from a center console. In addition,the steering unit 303 a may be a steering wheel.

A monitor device 11 which includes a display unit 8 capable ofoutputting various images and a voice output unit 9 capable ofoutputting various voices is provided in the vehicle room 2 a. Themonitor device 11 is provided on, for example, a central portion in thewidth direction (transverse direction) of the dashboard in the vehicleroom 2 a. In addition, the display unit 8 is configured with, forexample, a liquid crystal display (LCD) or an organic electroluminescentdisplay (OELD).

Here, an operation input unit 10 is provided on a display screen of thedisplay unit 8 as an area in which an image is displayed. For example,the operation input unit 10 is configured as a touch panel that maydetect the coordinates of a position where an indicator such as a fingeror a stylus approaches (including contact). Thus, the user (driver) mayview the image displayed on the display screen of the display unit 8 andmay execute various operation inputs by performing a touch (tap)operation using the indicator on the operation input unit 10.

In addition, in the embodiment, the operation input unit 10 may be anyof various physical interfaces such as a switch, a dial, a joystick, ora push button. Further, in the embodiment, another voice output devicemay be provided at a position different from the position of the monitordevice 11 in the vehicle room 2 a. In this case, various pieces of voiceinformation may be output from both the voice output unit 9 and theother voice output device. Further, in the embodiment, the monitordevice 11 may be configured to be able to display information related tovarious systems such as a navigation system and an audio system.

Further, as illustrated in FIGS. 1 and 2, the vehicle 1 according to theembodiment is configured as a four-wheel vehicle having two left andright front wheels 3F and two right and left rear wheels 3R.Hereinafter, for simplicity, the front wheels 3F and the rear wheels 3Rmay be collectively referred to as wheels. In the embodiment, the sideslip angles of some or all of the four wheels are changed (steered)according to an operation of the steering unit 303 a.

Further, the vehicle 1 is equipped with multiple (four in the exampleillustrated in FIGS. 1 and 2) in-vehicle cameras 15 a to 15 d as aperiphery monitoring imaging unit. The in-vehicle camera 15 a isprovided on a rear end 2 e of a vehicle body 2 (e.g., below a door 2 hof a rear trunk) and images the area behind the vehicle 1. Further, thein-vehicle camera 15 b is provided on a door mirror 2 g on a right end 2f of the vehicle body 2 and images the area at the right side of thevehicle 1. Further, the in-vehicle camera 15 c is provided on a frontend 2 c (e.g., a front bumper) of the vehicle body 2 and images the areain front of the vehicle 1. Further, the in-vehicle camera 15 d isprovided on the door mirror 2 g on a left end 2 d of the vehicle body 2and images the area at the left side of the vehicle 1. Hereinafter, forsimplicity, the in-vehicle cameras 15 a to 15 d may be collectivelyreferred to as an in-vehicle camera 15.

The in-vehicle camera 15 is a so-called digital camera having an imagingdevice such as a charge coupled device (CCD) or complementary metaloxide semiconductor (CMOS) image sensor (CIS). The in-vehicle camera 15performs imaging of the surroundings of the vehicle 1 at a predeterminedframe rate, and outputs image data of a captured image obtained by theimaging. The Image data obtained by the in-vehicle camera 15 mayconstitute a video image as a frame image.

Next, a system configuration provided to realize various controls in thevehicle 1 according to the embodiment will be described with referenceto FIG. 3. In addition, the system configuration illustrated in FIG. 3is merely an example and thus, may be set (changed) in various ways.

FIG. 3 is an exemplary and schematic block diagram illustrating a systemconfiguration of the vehicle 1 according to the embodiment. Asillustrated in FIG. 3, the vehicle 1 according to the embodimentincludes a braking system 301, an acceleration system 302, a steeringsystem 303, a transmission system 304, an obstacle sensor 305, atraveling state sensor 306, the in-vehicle camera 15, the monitor device11, a control device 310, and an in-vehicle network 350.

The braking system 301 controls the deceleration of the vehicle 1. Thebraking system 301 includes the braking unit 301 a, a braking controlunit 301 b, and a braking unit sensor 301 c.

The braking unit 301 a is, for example, a device for decelerating thevehicle 1 such as the brake pedal described above.

The braking control unit 301 b is configured as, for example, amicrocomputer having a hardware processor such as a central processingunit (CPU). The braking control unit 301 b controls the degree ofdeceleration of the vehicle 1 by driving an actuator (not illustrated)based on, for example, an instruction input via the in-vehicle network350 and operating the brake unit 301 a.

The braking unit sensor 301 c is a sensing device for detecting thestate of the braking unit 301 a. For example, when the braking unit 301a is configured as the brake pedal, the braking unit sensor 301 cdetects the position of the brake pedal or the pressure acting on thebrake pedal as the state of the braking unit 301 a. The braking unitsensor 301 c outputs the detected state of the braking unit 301 a to thein-vehicle network 350.

The acceleration system 302 controls the acceleration of the vehicle 1.The acceleration system 302 includes the acceleration unit 302 a, anacceleration control unit 302 b, and an acceleration unit sensor 302 c.

The acceleration unit 302 a is, for example, a device for acceleratingthe vehicle 1 such as the accelerator pedal described above.

The acceleration control unit 302 b is configured as a microcomputerhaving a hardware processor such as a CPU, for example. The accelerationcontrol unit 302 b controls the degree of acceleration of the vehicle 1by driving an actuator (not illustrated) based on, for example, aninstruction input via the in-vehicle network 350 and operating theacceleration unit 302 a.

The acceleration unit sensor 302 c is a sensing device for detecting thestate of the acceleration unit 302 a. For example, when the accelerationunit 302 a is configured as the accelerator pedal, the acceleration unitsensor 302 c detects the position of the accelerator pedal or thepressure acting on the accelerator pedal. The acceleration unit sensor302 c outputs the detected state of the acceleration unit 302 a to thein-vehicle network 350.

The steering system 303 controls the advancing direction of the vehicle1. The steering system 303 includes the steering unit 303 a, a steeringcontrol unit 303 b, and a steering unit sensor 303 c.

The steering unit 303 a is, for example, a device for steering thesteered wheels of the vehicle 1 such as the above-described steeringwheel or a handle.

The steering control unit 303 b is configured as, for example, amicrocomputer having a hardware processor such as a CPU. The steeringcontrol unit 303 b controls the advancing direction of the vehicle 1 bydriving an actuator (not illustrated) based on, for example, aninstruction input via the in-vehicle network 350 and operating thesteering unit 303 a.

The steering unit sensor 303 c is a sensing device for detecting thestate of the steering unit 303 a, i.e., a steering angle sensor fordetecting the steering angle of the vehicle 1. For example, when thesteering unit 303 a is configured as the steering wheel, the steeringunit sensor 303 c detects the position of the steering wheel or therotation angle of the steering wheel. In addition, in a case where thesteering unit 303 a is configured as the handle, the steering unitsensor 303 c may detect the position of the handle or the pressureacting on the handle. The steering unit sensor 303 c outputs thedetected state of the steering unit 303 a to the in-vehicle network 350.

The transmission system 304 controls the transmission ratio of thevehicle 1. The transmission system 304 includes the transmission unit304 a, a transmission control unit 304 b, and a transmission unit sensor304 c.

The transmission unit 304 a is, for example, a device for changing thetransmission ratio of the vehicle 1 such as the shift lever describedabove.

The transmission control unit 304 b is configured as, for example, acomputer having a hardware processor such as a CPU. The transmissioncontrol unit 304 b controls the transmission ratio of the vehicle 1 bydriving an actuator (not illustrated) based on, for example, aninstruction input via the in-vehicle network 350 and operating thetransmission unit 304 a.

The transmission unit sensor 304 c is a sensing device for detecting thestate of the transmission unit 304 a. For example, when the transmissionunit 304 a is configured as the shift lever, the transmission unitsensor 304 c detects the position of the shift lever or the pressureacting on the shift lever. The transmission unit sensor 304 c outputsthe detected state of the transmission unit 304 a to the in-vehiclenetwork 350.

The obstacle sensor 305 is a sensing device for detecting informationrelated to an object (obstacle) that may exist around the vehicle 1. Theobstacle sensor 305 includes, for example, a distance measurement sensorthat acquires the distance to an object existing around the vehicle 1.Examples of the distance measurement sensor include a sonar that obtainsthe distance by transmitting voice waves and receiving the voice wavesreflected by the object existing around the vehicle 1 or a laser radarthat obtains the distance by transmitting radio waves such as light andreceiving the radio waves reflected by the object existing around thevehicle 1. The obstacle sensor 305 outputs the detected information tothe in-vehicle network 350.

The traveling state sensor 306 is a device for detecting the travelingstate of the vehicle 1. The traveling state sensor 306 includes, forexample, a wheel speed sensor that detects the wheel speed of thevehicle 1, an acceleration sensor that detects the longitudinal ortransverse acceleration of the vehicle 1, or a gyro sensor that detectsthe turning speed (angular velocity) of the vehicle 1. The travelingstate sensor 306 outputs the detected traveling state to the in-vehiclenetwork 350.

The control device 310 is a device that comprehensively controls varioussystems provided in the vehicle 1. Although details will be describedlater, the control device 310 according to the embodiment has a functionof generating a peripheral image indicating a situation around thevehicle 1 based on image data as a result of imaging by the in-vehiclecamera 15 and displaying the generated peripheral image on the displayunit 8. In addition, the peripheral image mentioned here includes, forexample, a bird's eye view image obtained by looking down at thesituation around the vehicle 1 (including the vehicle 1) from above, athree-dimensional image obtained by three-dimensionally viewing thesituation around the vehicle 1 (including the vehicle 1) from anarbitrary viewpoint, or single camera image that is an image based onlyon one image data acquired from one in-vehicle camera 15.

The control device 310 is configured as an electronic control unit (ECU)including a central processing unit (CPU) 310 a, a read only memory(ROM) 310 b, a random access memory (RAM) 310 c, a solid state drive(SSD) 310 d, a display control unit 310 e, and a voice control unit 310f.

The CPU 310 a is a hardware processor that comprehensively controls thecontrol device 310. The CPU 310 a reads out various control programs(computer programs) stored in the ROM 310 b and the like, and realizesvarious functions according to instructions defined in the variouscontrol programs. In addition, the various control programs mentionedhere include a display control program for realizing a display controlprocessing of displaying the peripheral image described above, a realvehicle image, and a virtual vehicle image.

The ROM 310 b is a nonvolatile main storage device that storesparameters required for the execution of the various control programsdescribed above.

The RAM 310 c is a volatile main storage device that provides anoperating area for the CPU 310 a.

The SSD 310 d is a rewritable nonvolatile auxiliary storage device. Inaddition, the control device 310 according to the embodiment may beprovided with a hard disk drive (HDD) as an auxiliary storage deviceinstead of the SSD 310 d (or in addition to the SSD 310 d).

The display control unit 310 e mainly performs an image processing on acaptured image obtained from the in-vehicle camera 15 or the generationof image data to be output to the display unit 8 of the monitor device11 among various types of processings that may be executed by thecontrol device 310.

The voice control unit 310 f mainly performs, the generation of voicedata to be output to the voice output unit 9 of the monitor device 11among various processes that may be executed by the control device 310.

The in-vehicle network 350 interconnects the braking system 301, theacceleration system 302, the steering system 303, the transmissionsystem 304, the obstacle sensor 305, the traveling state sensor 306, theoperation input unit 10 of the monitor device 11, and the control device310 so as to enable communication therebetween.

By the way, various techniques have been conventionally studied tonotify the occupant of the situation around the vehicle 1 using theperipheral image described above, but in these techniques, for example,when the vehicle 1 moves toward a target position, it is necessary tonotify the occupant of the mode of movement of the vehicle 1 to thetarget position along with the situation around the vehicle 1 in aneasy-to-understand manner.

Accordingly, by realizing, in the control device 310, a display controldevice 400 having functions illustrated in FIG. 4, the embodimentrealizes to notify the occupant of the mode of movement of the vehicle 1to the target position along with the situation around the vehicle 1 inan easy-to-understand manner.

FIG. 4 is an exemplary and schematic block diagram illustratingfunctions of the display control device 400 according to the embodiment.The functions illustrated in FIG. 4 are realized in the control device310 by cooperation of software and hardware. That is, the functionsillustrated in FIG. 4 are realized as a result of the CPU 310 a of thecontrol device 310 reading out and executing a predetermined controlprogram (camera parameter estimation program) stored in the ROM 310 band the like. In addition, in the embodiment, some of the functionsillustrated in FIG. 4 may be realized by dedicated hardware (circuits).

As illustrated in FIG. 4, the display control device 400 according tothe embodiment includes an operation detection unit 401, a proxy controlunit 402, an image data acquisition unit 403, an image generation unit404, and a display processing unit 405.

The operation detection unit 401 detects various operations executed onthe vehicle 1. For example, the operation detection unit 401 detects anoperation (such as a touch operation) input via the operation input unit10 of the monitor device 11.

The proxy control unit 402 executes proxy control of executing at leasta part of a driving operation of the vehicle 1 by the driver byappropriately controlling the braking system 301, the accelerationsystem 302, the steering system 303, and the transmission system 304described above. According to the proxy control, the movement of thevehicle 1 to the target position may be realized automatically (orsemi-automatically) without depending on only the driving operation bythe driver. In addition, examples of the proxy control include automaticparking control and the like. As the target position that may be set inthe automatic parking control, a parking position where the vehicle 1finally arrives or a quick-turn position where there is a possibility ofthe vehicle 1 temporarily stopping before reaching the parking positionis conceivable. In the embodiment, the proxy control unit 402 may berealized in a dedicated ECU that comprehensively controls the travelingcontrol of the vehicle 1, separately from the control device 310 thatconstitutes the display control device 400. In addition, it goes withoutsaying that the technique of the embodiment may also be used for proxycontrol other than automatic parking control.

The image data acquisition unit 403 acquires, from the in-vehicle camera15, image data as a result of imaging by the in-vehicle camera 15.

The image generation unit 404 generates an image to be displayed on thedisplay unit 8 of the monitor device 11 based on the image data acquiredby the image data acquisition unit 403.

The display processing unit 405 controls the display content of thedisplay unit 8 of the monitor device 11 based on the image generated bythe image generation unit 404. More specifically, when the vehicle 1moves toward the target position, the display processing unit 405displays a peripheral image indicating the situation around the vehicle1 in a display mode illustrated in, for example, FIG. 5 on the displayunit 8 and displays (superimposes) a real vehicle image indicating thecurrent state of the vehicle 1 and a virtual vehicle image indicting thefuture state of the vehicle 1 on the peripheral image.

FIG. 5 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image according to the embodiment. In addition, hereinafter,an example in which a three-dimensional image is used as a peripheralimage will be mainly described, but, in the embodiment, as describedabove, the peripheral image also includes a bird's eye view image or asingle camera image, in addition to the three-dimensional image. Thus,in the embodiment, the real vehicle image and the virtual vehicle imagemay also be superimposed on the bird's eye view image or the singlecamera image.

As illustrated in FIG. 5, in the embodiment, the display processing unit405 displays, on the display unit 8, an image IM500 including, forexample, an image 500 representing a three-dimensional image as anexample of a peripheral image, an image 501 representing a real vehicleimage indicating the current state (e.g., position, orientation, orshape) of the vehicle 1 at a viewpoint corresponding to the image 500,and an image 502 representing a virtual vehicle image indicating thefuture state of the vehicle 1 (when the movement of the vehicle 1proceeds for a certain time or by a certain distance) at a viewpointcorresponding to the image 500.

In addition, in the embodiment, the display processing unit 405 maygenerate the real vehicle image and the virtual vehicle image afterdifferentiating, for example, the shape, color, brightness, ortransmittance thereof such that the virtual vehicle image is notconfused with information indicating the current state of the vehicle 1.For example, as in the example illustrated in FIG. 5, the displayprocessing unit 405 may generate the image 501 representing the realvehicle image based on the overall shape of the vehicle 1 includingwheels, and may generate the image 502 representing the virtual vehicleimage based on the shape of the vehicle body 2 of the vehicle 1including no wheels.

Furthermore, although details will be described later, in theembodiment, the display processing unit 405 may display a real vehicleimage and a virtual vehicle image, for example, as 3D polygons. In thiscase, the display processing unit 405 may selectively switch betweendisplaying an overlapping portion between a real vehicle image and avirtual vehicle image in a display mode in which either one of the realvehicle image and the virtual vehicle image is identifiable by executinga hidden surface removal processing on the overlapping portion anddisplaying the overlapping portion between the real vehicle image andthe virtual vehicle image in a display mode in which both the realvehicle image and the virtual vehicle image are identifiable byexecuting no hidden surface removal processing on the overlappingportion.

According to the above-described peripheral image on which the realvehicle image and the virtual vehicle image are superimposed, since thecurrent state of the vehicle 1 and the situation around the vehicle 1may be notified, including the future state of the vehicle 1, to theoccupant in an easy-to-understand manner, it is possible to notify theoccupant of the mode of movement of the vehicle 1 to the target positionalong with the situation around the vehicle 1 in an easy-to-understandmanner. However, there is room for an improvement in setting the displayposition of the virtual vehicle image with respect to the real vehicleimage in order to further pursue the ease of understanding ofnotification.

Accordingly, in the embodiment, when the vehicle 1 moves toward a targetposition, the display processing unit 405 changes a relationship betweenthe display position of a real vehicle image and the display position ofa virtual vehicle image according to the remaining distance of thevehicle 1 to the target position. In addition, the remaining distancemay be calculated in consideration of the traveling distance of thevehicle 1 estimated using the detection results of various sensorsprovided in the vehicle 1.

For example, in the embodiment, as illustrated in FIGS. 6 and 7, thedisplay processing unit 405 maintains a substantially constant distancebetween the display position of a real vehicle image and the displayposition of a virtual vehicle image when the remaining distance to atarget position exceeds a threshold value (or is greater than or equalto the threshold value).

FIG. 6 is an exemplary and schematic image diagram for explaining anexample of a relationship between the display position of a real vehicleimage and the display position of a virtual vehicle image in theembodiment. In the example illustrated in FIG. 6, a FIG. 601 correspondsto a real vehicle image, a position P601 corresponds to the displayposition of the real vehicle image, a FIG. 602 corresponds to a virtualvehicle image, and a position P602 of the FIG. 602 corresponds to thedisplay position of the virtual vehicle image, and a position P610corresponds to a target position.

As illustrated in FIG. 6, the positions P601, P602, and P610 arearranged on an arc C600 centered on a position P600. In addition, in theembodiment, the arc C600 (and the position P600) may be acquired basedon the course of the vehicle 1 that is systematically and appropriatelycalculated in proxy control (e.g., automatic parking control) executedby the proxy control unit 402, or may be acquired based on the course ofthe vehicle 1 that is calculated based on the detection result of thesteering unit sensor 303 c as a steering angle sensor that detects thesteering angle of the vehicle 1. Accordingly, in the embodiment, themovement of the vehicle 1 to the target position may be realized underthe proxy control by the proxy control unit 402, or may be realizedunder a manual operation by the driver.

Here, in the embodiment, when the remaining distance (from the positionP601) to the position P610 corresponding to the target position exceedsthe threshold value, the distance along the arc C600 between theposition P601 corresponding to the display position of the real vehicleimage and the position P602 corresponding to the display position of thevirtual vehicle image is kept substantially constant. Thus, the displayprocessing unit 405 displays the peripheral image including the realvehicle image and the virtual vehicle image in a display modeillustrated in FIG. 7.

FIG. 7 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image, corresponding to the example illustrated in FIG. 6, inthe embodiment. In the embodiment, the display processing unit 405displays an image IM700 illustrated in FIG. 7 on the display unit 8 in acase where a relationship between the display position of a real vehicleimage and a target position is as in the example illustrated in FIG. 6.

As illustrated in FIG. 7, the image IM700 includes an image 700representing a three-dimensional image as an example of a peripheralimage, an image 701 representing a real vehicle image superimposed on aposition P701 in the image 700, and an image 702 representing a virtualvehicle image superimposed on a position P702 in the image 700 which isspaced apart from the position P701 by a certain distance along anassumed course of the vehicle 1. In addition, in the image IM700illustrated in FIG. 7, an icon 710 indicating a target position is alsodisplayed at the position P701 corresponding to the target position.

The display mode as described above in which the image 701 representingthe real vehicle image and the image 702 representing the virtualvehicle image are displayed as being spaced apart from each other by acertain distance is continued as long as the remaining distance to thetarget position exceeds a threshold value. Thus, the occupant may easilyrecognize the remaining distance to the target position by visuallyconfirming the state of separation between the image 701 representingthe real vehicle image and the image 702 representing the virtualvehicle image.

In addition, in the example illustrated in FIG. 7, the distance betweenthe display positions of the real vehicle image and the virtual vehicleimage is set such that the image 702 representing the virtual vehicleimage overlaps with at least a portion of the image 701 representing thereal vehicle image, but, in the embodiment, the distance between thedisplay positions of the real vehicle image and the virtual vehicleimage may be set such that the image 702 representing the virtualvehicle image and the image 701 representing the real vehicle image areseparated from each other without overlapping.

Meanwhile, in the embodiment, as illustrated in FIGS. 8 and 9, thedisplay processing unit 405 changes the distance between the displayposition of a real vehicle image and the display position of a virtualvehicle image according to the remaining distance to a target positionwhen the remaining distance is below the threshold value (or is equal toor less than the threshold value).

FIG. 8 is an exemplary and schematic image diagram for explaining anexample of a relationship between the display position of a real vehicleimage and the display position of a virtual vehicle image, differentfrom FIG. 6, in the embodiment. In the example illustrated in FIG. 8, aFIG. 801 corresponds to a real vehicle image, a position P801corresponds to the display position of the real vehicle image, a FIG.802 corresponds to a virtual vehicle image, a position P802 of the FIG.802 corresponds to the display position of the virtual vehicle image,and a position P810 corresponds to a target position. In addition, inthe example illustrated in FIG. 8, the position P802 and the positionP810 coincide with each other.

As illustrated in FIG. 8, the positions P801, P802, and P810 arearranged on an arc C800 centered on a position P800. In addition, in theexample illustrated in FIG. 8, the arc C800 (and the position P800) maybe acquired based on the course of the vehicle 1 that is systematicallyand appropriately calculated in proxy control, or may be acquired basedon the course of the vehicle 1 that is calculated based on the detectionresult of the steering unit sensor 303 c, similarly to the exampleillustrated in FIG. 6.

Here, in the example illustrated in FIG. 8, the remaining distance (fromthe position P801) to the position P810 corresponding to the targetposition is below the threshold value as a result of being reducedcompared to the example illustrated in FIG. 6. In such a case, thedisplay processing unit 405 changes the distance between the displayposition of the real vehicle image and the display position of thevirtual vehicle image according to the remaining distance to the targetposition, rather than maintaining a constant distance between thedisplay position of the real vehicle image and the display position ofthe virtual vehicle image.

In other words, in the embodiment, the display processing unit 405maintains a constant distance between the display position of thevirtual vehicle image and the display position of the real vehicle imageuntil the display position of the virtual vehicle image reaches thetarget position, for example, when the remaining distance to the targetposition exceeds the threshold value, and changes the distance betweenthe display position of the virtual vehicle image and the displayposition of the real vehicle image according to the remaining distancesuch that, for example, a state where the display position of thevirtual vehicle image has reached the target position is maintained whenthe remaining distance to the target position is below the thresholdvalue. Thus, the display processing unit 405 displays the peripheralimage including the real vehicle image and the virtual vehicle image ina display mode illustrated in FIG. 9.

FIG. 9 is an exemplary and schematic diagram illustrating an example ofa real vehicle image and a virtual vehicle image displayed on aperipheral image, corresponding to the example illustrated in FIG. 8, inthe embodiment. In the embodiment, the display processing unit 405displays an image IM900 illustrated in FIG. 9 on the display unit 8 in acase where a relationship between the display position of a real vehicleimage and a target position is as in the example illustrated in FIG. 8.

As illustrated in FIG. 9, the image IM900 includes an image 900representing a three-dimensional image as an example of a peripheralimage, an icon 910 indicating a target position superimposed on aposition P910 in the image 900, an image 901 representing a real vehicleimage superimposed on a position P901 in the image 900, and an image 902representing a virtual vehicle image superimposed on a position P902 inthe image 900 which coincides with the position P910 of the icon 910indicating the target position.

In the example illustrated in FIG. 9, the distance between the image 901representing the real vehicle image and the image 902 representing thevirtual vehicle image gradually decreases as the vehicle 1 moves. Morespecifically, as the vehicle 1 moves, the image 901 representing thereal vehicle image gradually approaches the image 902 representing thevirtual vehicle image which is fixedly displayed at the position P910 asthe target position. Thus, the occupant may easily recognize the mode ofmovement of the vehicle 1 to the target position by visually confirmingthe mode of approach of the image 901 representing the real vehicleimage and the image 902 representing the virtual vehicle image.

In addition, in the embodiment, the timing at which the remainingdistance to the target position reaches the threshold value and thetiming at which the display position of the virtual vehicle imagereaches the target position may not necessarily coincide with eachother. For example, in the embodiment, a method of maintaining aconstant distance between the display position of a real vehicle imageand the display position of a virtual vehicle image until the displayposition of the virtual vehicle image reaches a position spaced forwardapart from a target position by a predetermined distance and thereafter,gradually reducing the distance between the display position of the realvehicle image and the display position of the virtual vehicle imageaccording to the remaining distance such that the timing at which thedisplay position of the real vehicle image reaches the target positionand the timing at which the display position of the virtual vehicleimage reaches the target position coincide with each other is alsoconceivable.

Incidentally, in the embodiment, as described above, the real vehicleimage and the virtual vehicle image may be generated afterdifferentiating the shape, color, brightness, or transmittance thereof.Thus, in the embodiment, the virtual vehicle image may be displayed in astate where the content (background) other than the vehicle 1 displayedin the peripheral image is viewed through the virtual vehicle image. Insuch a case, when the colors or brightnesses of the real vehicle image,the virtual vehicle image, and the peripheral image are similar to eachother, the differentiability of each image is deteriorated.

Accordingly, in the embodiment, the display processing unit 405 improvesthe differentiability of each of a real vehicle image, a virtual vehicleimage, and a peripheral image using one or a combination of the methodsillustrated in FIGS. 10 and 11.

First, a method of improving the differentiability of a virtual vehicleimage with respect to a real vehicle image will be described. In theembodiment, the display processing unit 405 may improve thedifferentiability of a virtual vehicle image with respect to a realvehicle image by displaying the real vehicle image and the virtualvehicle image in a display mode like an image IM1000 illustrated in FIG.10.

FIG. 10 is an exemplary and schematic diagram illustrating an example ofa display mode of a real vehicle image and a virtual vehicle image on aperipheral image according to the embodiment. In the example illustratedin FIG. 10, the image IM1000 is configured by superimposing an image1001 representing a real vehicle image and an image 1002 representing avirtual vehicle image on an image 1000 representing a three-dimensionalimage as an example of a peripheral image. Further, the image 1001representing the real vehicle image and the image 1002 representing thevirtual vehicle image are displayed as 3D polygons which partiallyoverlap each other.

As illustrated in FIG. 10, in the embodiment, the display processingunit 405 may display a real vehicle image and a virtual vehicle image ina display mode in which an overlapping portion between the real vehicleimage and the virtual vehicle image is identifiable. More specifically,the display processing unit 405 does not execute a hidden surfaceremoval processing on the overlapping portion between the real vehicleimage and the virtual vehicle image, so that the overlapping portion maybe displayed in a display mode in which both the real vehicle image andthe virtual vehicle image are identifiable. According to such a displaymode, for example, even when the difference in color (and/or brightness)between a real vehicle image and a virtual vehicle image is relativelysmall, it is possible to make it easy to recognize the positionaldeviation between the real vehicle image and the virtual vehicle image.

However, for example, when the difference in color (and/or brightness)between a real vehicle image and a virtual vehicle image is relativelylarge, there may be no major inconvenience even if the hidden surfaceremoval processing is executed on an overlapping portion between thereal vehicle image and the virtual vehicle image. Accordingly, in theembodiment, the display processing unit 405 may selectively switchbetween displaying an overlapping portion between a real vehicle imageand a virtual vehicle image in a display mode in which either one of thereal vehicle image and the virtual vehicle image is identifiable byexecuting a hidden surface removal processing on the overlapping portionand displaying the overlapping portion between the real vehicle imageand the virtual vehicle image in a display mode in which both the realvehicle image and the virtual vehicle image are identifiable byexecuting no hidden surface removal processing on the overlappingportion.

Next, a method of improving the differentiability of a virtual vehicleimage with respect to a peripheral image will be described. In theembodiment, the display processing unit 405 may improve thedifferentiability of a virtual vehicle image with respect to aperipheral image by appropriately adjusting the color (and/orbrightness) of the virtual vehicle image in a display mode like an imageIM1100 illustrated in FIG. 11.

FIG. 11 is an exemplary and schematic diagram illustrating an example ofadjustment of the color (and/or brightness) of a virtual vehicle imagethat may be executed in the embodiment. In the example illustrated inFIG. 11, the image IM1100 is configured by superimposing an image 1101representing a real vehicle image and an image 1102 representing avirtual vehicle image on an image 1100 representing a three-dimensionalimage as an example of a peripheral image.

As illustrated in FIG. 11, in the embodiment, when the difference incolor (and/or brightness) between a virtual vehicle image and aperipheral portion of a peripheral image around the virtual vehicleimage is relatively small, the display processing unit 405 may adjustthe color (and/or brightness) of the virtual vehicle image such that thedifference becomes, for example, a certain level or more. According tosuch a display mode, for example, even when the difference in color(and/or brightness) between the virtual vehicle image and the peripheralportion is relatively small, it is possible to clearly differentiate thevirtual vehicle image and the peripheral portion.

In addition, in the example illustrated in FIG. 11, a hidden surfaceremoval processing is executed on an overlapping portion of the image1101 representing the real vehicle image and the image 1102 representingthe virtual vehicle image, but it goes without saying that the hiddensurface removal processing needs not to be executed on the overlappingportion. In the latter case, the overall color (and/or brightness) ofthe virtual vehicle image including the overlapping portion with thereal vehicle image may be adjusted collectively.

Further, the adjustment of the color of the virtual vehicle image asdescribed above may also be used to notify a driver's driving operationerror when the vehicle 1 moves by manual driving, in addition todifferentiation between the virtual vehicle image and the peripheralportion. The display position of the virtual vehicle image is determinedbased on the detection result of the steering unit sensor 303 c as asteering angle sensor that detects the steering angle of the vehicle 1,for example, when the vehicle 1 moves by manual driving, but, when thedetermined display position is out of the course to be reached to thetarget position, changing the color of the virtual vehicle image toalert the driver is useful because it may notify a driver's drivingoperation error.

By the way, when the color (and/or brightness) of the virtual vehicleimage is appropriately adjusted by the method illustrated in FIG. 11, itis assumed that the difference in color (and/or brightness) between thevirtual vehicle image and the real vehicle image becomes relativelysmall and the differentiability of the real vehicle image with respectto the virtual vehicle image is deteriorated.

Accordingly, in the embodiment, the display processing unit 405 mayachieve both an improvement in the differentiability of a virtualvehicle image with respect to a peripheral image and an improvement inthe differentiability of a real vehicle image with respect to thevirtual vehicle image by appropriately adjusting the color (and/orbrightness) of the real vehicle image in a display mode like an imageIM1200 illustrated in FIG. 12.

FIG. 12 is an exemplary and schematic diagram illustrating an example ofadjustment of the color (and/or brightness) of a real vehicle image thatmay be executed in the embodiment. In the example illustrated in FIG.12, the image IM1200 is configured by superimposing an image 1201representing a real vehicle image and an image 1202 representing avirtual vehicle image on an image 1200 representing a three-dimensionalimage as an example of a peripheral image.

As illustrated in FIG. 12, in the embodiment, when the difference incolor (and/or brightness) between a virtual vehicle image and a realvehicle image becomes relatively small as a result of adjustment of thecolor (and/or brightness) of the virtual vehicle image, the displayprocessing unit 405 may adjust the color (and/or brightness) of the realvehicle image such that the difference becomes, for example, a certainlevel or more. According to such a display mode, it is possible toclearly differentiate the virtual vehicle image and the real vehicleimage.

In addition, as in the example illustrated in FIG. 11, in the exampleillustrated in FIG. 12, a hidden surface removal processing is executedon an overlapping portion between the image 1201 representing the realvehicle image and the image 1202 representing the virtual vehicle image,but it goes without saying that the hidden surface removal processingneeds not to be executed on the overlapping portion.

Hereinafter, a processing executed by the display control device 400according to the embodiment will be described with reference to FIG. 13.

FIG. 13 is an exemplary and schematic flowchart illustrating a series ofprocessings executed by the display control device 400 according to theembodiment to display a real vehicle image and a virtual vehicle image.The series of processings illustrated in FIG. 13 are repeatedly executedunder a situation where monitoring the periphery of the vehicle 1 isrequired.

As illustrated in FIG. 13, in the embodiment, first, in step S1301, theimage data acquisition unit 403 of the display control device 400acquires image data as a result of imaging by the in-vehicle camera 15.

Then, in step S1302, the image generation unit 404 of the displaycontrol device 400 generates a peripheral image as one of images to bedisplayed on the display unit 8. As described above, the peripheralimage includes a three-dimensional image, a bird's eye view image, asingle camera image, or the like.

Then, in step S1303, the image generation unit 404 of the displaycontrol device 400 generates a real vehicle image and a virtual vehicleimage to be superimposed on the peripheral image. As described above,the real vehicle image is an image indicating the current state of thevehicle 1, and the virtual vehicle image is an image indicating thefuture state of the vehicle 1. These images are calculated based on thedetection results of various sensors provided in the vehicle 1 or thecontent of proxy control executed by the proxy control unit 402.

Then, in step S1304, the display processing unit 405 of the displaycontrol device 400 determines the display positions of the real vehicleimage and the virtual vehicle image based on the remaining distance to atarget position. More specifically, the display processing unit 405maintains a substantially constant distance between the display positionof the real vehicle image and the display position of the virtualvehicle image when the remaining distance to the target position exceedsa threshold value, and changes the distance between the display positionof the real vehicle image and the display position of the virtualvehicle image according to the remaining distance when the remainingdistance to the target position is below the threshold value.

Then, in step S1305, the display processing unit 405 of the displaycontrol device 400 outputs the peripheral image on which the realvehicle image and the virtual vehicle image are superimposed to thedisplay unit 8. Then, the processing is terminated.

As described above, the display control device 400 according to theembodiment includes the image data acquisition unit 403 and the displayprocessing unit 405. The image data acquisition unit 403 acquires imagedata as a result of imaging by the in-vehicle camera 15 as an imagingunit that images the situation around the vehicle 1. When the vehicle 1moves toward the target position, the display processing unit 405displays, on the display unit 8, a peripheral image indicating thesituation around the vehicle 1 generated based on the image data, anddisplays, on the peripheral image, a real vehicle image as a firstvehicle image indicating the current state of the vehicle 1 and avirtual vehicle image as a second vehicle image indicating the futurestate of the vehicle 1. Then, the display processing unit 405 changes arelationship between the display position of the real vehicle image andthe display position of the virtual vehicle image according to theremaining distance to the target position of the vehicle 1.

According to the configuration as described above, it is possible tonotify the occupant of the mode of movement of the vehicle 1 to thetarget position along with the situation around the vehicle 1 in aneasy-to-understand manner by the peripheral image on which the realvehicle image and the virtual vehicle image, the display positions ofwhich change according to the remaining distance to the target position,are displayed.

In the display control device 400 according to the embodiment, thedisplay processing unit 405 maintains a substantially constant distancebetween the display position of the real vehicle image and the displayposition of the virtual vehicle image when the remaining distanceexceeds a threshold value, and changes the distance between the displayposition of the real vehicle image and the display position of thevirtual vehicle image according to the remaining distance when theremaining distance is below the threshold value. According to thisconfiguration, it is possible to notify the occupant of the degree ofapproach of the vehicle 1 to the target position in aneasy-to-understand manner based on whether or not there is a change inthe distance between the display position of the real vehicle image andthe display position of the virtual vehicle image.

In addition, in the display control device 400 according to theembodiment, the display processing unit 405 may display the virtualvehicle image on a virtual course in the peripheral image calculatedbased on a virtual arc that interconnects the current position of thevehicle 1 and the target position. According to this configuration, itis possible to display the virtual vehicle image at an appropriateposition corresponding to the future of the vehicle 1 in considerationof the virtual arc that interconnects the current position of thevehicle 1 and the target position.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 may display the virtual vehicle image ona virtual course in the peripheral image calculated based on a detectionresult of the steering unit sensor 303 c as a steering angle sensor thatdetects a steering angle of the vehicle 1. According to thisconfiguration, it is possible to display the virtual vehicle image at anappropriate position corresponding to the future of the vehicle 1 inconsideration of the detection result of the steering angle sensor.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 may display the virtual vehicle image soas to overlap with at least a portion of the real vehicle image.According to this configuration, it is possible to prevent the virtualvehicle image from being misidentified as an image indicating the stateof another vehicle.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 may display the real vehicle image andthe virtual vehicle image in a display mode in which an overlappingportion between the real vehicle image and the virtual vehicle image isidentifiable. More specifically, the display processing unit 405 maydisplay the real vehicle image and the virtual vehicle image withoutexecuting a hidden surface removal processing on the overlapping portionbetween the real vehicle image and the virtual vehicle image whendisplaying the real vehicle image and the virtual vehicle image as 3Dpolygons. According to this configuration, it is possible to make iteasy to identify the positional deviation between the real vehicle imageand the virtual vehicle image.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 may adjust at least one of the color andthe brightness of the virtual vehicle image according to at least one ofthe difference between the color of the virtual vehicle image and thecolor of a peripheral portion of the peripheral image around the virtualvehicle image and the difference between the brightness of the virtualvehicle image and the brightness of the peripheral portion. According tothis configuration, it is possible to make it easy to identify thevirtual vehicle image and the peripheral portion, for example, even whenthe difference in color and/or brightness between the virtual vehicleimage and the peripheral image (peripheral portion) is relatively small.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 generates the real vehicle image basedon the overall shape of the vehicle 1 including wheels, and generatesthe virtual vehicle image based on the shape of the vehicle body 2 ofthe vehicle 1 including no wheels. According to this configuration, itis possible to make it easy to identify that the virtual vehicle imagedoes not indicate the current state of the vehicle 1.

Further, in the display control device 400 according to the embodiment,the display processing unit 405 may change a relationship between thedisplay position of the real vehicle image and the display position ofthe virtual vehicle image according to the remaining distance when thevehicle 1 moves toward the target position under proxy control thatexecutes at least a part of a driving operation of the vehicle by thedriver. According to this configuration, it is possible to notify theoccupant of the mode of movement of the vehicle 1 to the target positionunder the proxy control along with the situation around the vehicle 1 inan easy-to-understand manner.

Further, in the display control device 400 according to the embodiment,the target position is a parking position where the vehicle 1 finallyreaches in automatic parking control as the proxy control or aquick-turn position where there is a possibility of the vehicle 1temporarily stopping before reaching the parking position. According tothis configuration, it is possible to notify the occupant of the mode ofmovement of the vehicle 1 to the parking position or the quick-turnposition along with the situation around the vehicle 1 in aneasy-to-understand manner.

In addition, a display control program executed in the control device310 according to the embodiment may be provided or distributed via anetwork such as the Internet. In other words, the display controlprogram executed in the control device 310 according to the embodimentmay be provided in a form in which it accepts a download via a networksuch as the Internet in a state of being stored on a computer connectedto the network.

<Modifications>

In the embodiment described above, as an example, a technique ofchanging a relationship between the display position of a real vehicleimage and the display position of a virtual vehicle image according tothe remaining distance to a target position is illustrated. However, thetechnique disclosed here is not limited to the technique of changing therelationship between the display positions of the real vehicle image andthe virtual vehicle image according to the remaining distance, and alsoincludes a technique of changing a display mode other than the displayposition, for example, the color, brightness, or transmittance of atleast one of the real vehicle image and the virtual vehicle imageaccording to the remaining distance. By the latter technique, as in theformer technique, it is possible to notify the occupant of the mode ofmovement of a vehicle to a target position along with the situationaround the vehicle in an easy-to-understand manner by changing thedisplay mode of at least one of the real vehicle image and the virtualvehicle image on the peripheral image according to the remainingdistance.

For example, as an example of a technique of changing the coloraccording to the remaining distance, a technique of gradually changingthe color of a virtual vehicle image (and/or a real vehicle image) fromgray to white as the remaining distance decreases is conceivable.Further, as an example of a technique of changing the brightnessaccording to the remaining distance, a technique of gradually changingthe brightness of a virtual vehicle image (and/or a real vehicle image)from a dark state to a bright state as the remaining distance decreasesis conceivable. Further, as an example of a technique of changing thetransmittance according to the remaining distance, a technique ofgradually changing the transmittance of a virtual vehicle image (and/ora real vehicle image) from a high value to a low value as the remainingdistance decreases is conceivable. Each of these three techniques may beused alone, or may be used as a combination of one or more of fourtechniques including the technique as in the above-described embodimentof changing the display position according to the remaining distance.

A display control device as an aspect of this disclosure includes animage data acquisition unit configured to acquire image data as a resultof imaging by an imaging unit that images a situation around a vehicleand a display processing unit configured to display, on a display unit,a peripheral image indicating the situation around the vehicle generatedbased on the image data and also display, on the peripheral image, afirst vehicle image indicating a current state of the vehicle and asecond vehicle image indicating a future state of the vehicle when thevehicle moves toward a target position and to change a display mode ofat least one of the first vehicle image and the second vehicle imageaccording to a remaining distance to the target position of the vehicle.

According to the display control device described above, it is possibleto notify an occupant of the mode of movement of the vehicle to thetarget position along with the situation around the vehicle in aneasy-to-understand manner by changing the display mode of at least oneof the first vehicle image and the second vehicle image according to theremaining distance.

In the display control device described above, the display processingunit may execute a change in a relationship between a display positionof the first vehicle image and a display position of the second vehicleimage as a change in the display mode of at least one of the firstvehicle image and the second vehicle image. According to thisconfiguration, it is possible to visually express the mode of movementof the vehicle in an easy-to-understand manner by changing therelationship between the display position of the first vehicle image andthe display position of the second vehicle image according to theremaining distance.

In this case, the display processing unit may maintain a substantiallyconstant distance between the display position of the first vehicleimage and the display position of the second vehicle image when theremaining distance exceeds a threshold value, and change the distancebetween the display position of the first vehicle image and the displayposition of the second vehicle image according to the remaining distancewhen the remaining distance is below the threshold value. According tothis configuration, it is possible to notify the occupant of the degreeof approach of the vehicle to the target position in aneasy-to-understand manner based on whether or not there is a change inthe distance between the display position of the first vehicle image andthe display position of the second vehicle image.

Further, in the display control device described above, the displayprocessing unit may display the second vehicle image on a virtual coursein the peripheral image calculated based on a virtual arc thatinterconnects a current position of the vehicle and the target position.According to this configuration, it is possible to display the secondvehicle image at an appropriate position corresponding to the future ofthe vehicle in consideration of the virtual arc that interconnects thecurrent position of the vehicle and the target position.

Further, in the display control device described above, the displayprocessing unit may display the second vehicle image on a virtual coursein the peripheral image calculated based on a detection result of asteering angle sensor that detects a steering angle of the vehicle.According to this configuration, it is possible to display the secondvehicle image at an appropriate position corresponding to the future ofthe vehicle in consideration of the detection result of the steeringangle sensor.

Further, in the display control device described above, the displayprocessing unit may display the second vehicle image so as to overlapwith at least a portion of the first vehicle image. According to thisconfiguration, it is possible to prevent the second vehicle image frombeing misidentified as an image indicating the state of another vehicle.

In this case, the display processing unit may display the first vehicleimage and the second vehicle image in a display mode in which anoverlapping portion between the first vehicle image and the secondvehicle image is identifiable. According to this configuration, it ispossible to easily identify the positional deviation between the firstvehicle image and the second vehicle image.

Further, in this case, the display processing unit may display the firstvehicle image and the second vehicle image without executing a hiddensurface removal processing on the overlapping portion between the firstvehicle image and the second vehicle image when displaying the firstvehicle image and the second vehicle image as 3D polygons. According tothis configuration, it is possible to make it easy to identify thepositional deviation between the first vehicle image and the secondvehicle image.

In addition, in the display control device described above, the displayprocessing unit may adjust at least one of a color and a brightness ofthe second vehicle image according to at least one of a differencebetween the color of the second vehicle image and a color of aperipheral portion of the peripheral image around the second vehicleimage and a difference between the brightness of the second vehicleimage and a brightness of the peripheral portion. According to thisconfiguration, it is possible to make it easy to identify the secondvehicle image and the peripheral portion, for example, even when thedifference in color and/or brightness between the second vehicle imageand the peripheral image (peripheral portion) is relatively small.

Further, in the display control device described above, the displayprocessing unit may generate the first vehicle image based on an overallshape of the vehicle including a wheel, and generates the second vehicleimage based on a shape of a vehicle body of the vehicle including nowheel. According to this configuration, it is possible to make it easyto identify that the second vehicle image does not indicate the currentstate of the vehicle.

In the display control device described above, the display processingunit may change the display mode of at least one of the first vehicleimage and the second vehicle image according to the remaining distancewhen the vehicle moves toward the target position under proxy controlthat executes at least a part of a driving operation of the vehicle by adriver. According to this configuration, it is possible to notify theoccupant of the mode of movement of the vehicle to the target positionunder the proxy control along with the situation around the vehicle inan easy-to-understand manner.

In this case, the target position may be a parking position where thevehicle finally reaches in automatic parking control as the proxycontrol or a quick-turn position where there is a possibility of thevehicle temporarily stopping before reaching the parking position.According to this configuration, it is possible to notify the occupantof the mode of movement of the vehicle to the parking position or thequick-turn position along with the situation around the vehicle in aneasy-to-understand manner.

Although the embodiments and modifications disclosed here have beenexemplified above, the above-described embodiments and modificationsthereof are merely given by way of example, and are not intended tolimit the scope of this disclosure. Such novel embodiments andmodifications may be implemented in various other modes, and variousomissions, substitutions, combinations, and changes thereof may be madewithout departing from the gist of this disclosure. In addition, theembodiments and modifications may be included in the scope and gist ofthis disclosure and are included in the disclosure described in theclaims and the equivalent scope thereof.

The principles, preferred embodiment and mode of operation of thepresent invention have been described in the foregoing specification.However, the invention which is intended to be protected is not to beconstrued as limited to the particular embodiments disclosed. Further,the embodiments described herein are to be regarded as illustrativerather than restrictive. Variations and changes may be made by others,and equivalents employed, without departing from the spirit of thepresent invention. Accordingly, it is expressly intended that all suchvariations, changes and equivalents which fall within the spirit andscope of the present invention as defined in the claims, be embracedthereby.

What is claimed is:
 1. A display control device comprising: an imagedata acquisition unit configured to acquire image data as a result ofimaging by an imaging unit that images a situation around a vehicle; anda display processing unit configured to display, on a display unit, aperipheral image indicating the situation around the vehicle generatedbased on the image data and also display, on the peripheral image, afirst vehicle image indicating a current state of the vehicle and asecond vehicle image indicating a future state of the vehicle when thevehicle moves toward a parking target position, wherein the displayprocessing unit maintains a constant distance between the displayposition of the first vehicle image and the display position of thesecond vehicle image when a remaining distance to the parking targetposition of the vehicle exceeds a threshold value, and changes thedistance between the display position of the first vehicle image and thedisplay position of the second vehicle image according to the remainingdistance when the remaining distance to the parking target position ofthe vehicle is below the threshold value.
 2. The display control deviceaccording to claim 1, wherein the display processing unit displays thesecond vehicle image on a virtual course in the peripheral imagecalculated based on a virtual arc that interconnects a current positionof the vehicle and the target position.
 3. The display control deviceaccording to claim 1, wherein the display processing unit displays thesecond vehicle image on a virtual course in the peripheral imagecalculated based on a detection result of a steering angle sensor thatdetects a steering angle of the vehicle.
 4. The display control deviceaccording to claim 1, wherein the display processing unit displays thesecond vehicle image so as to overlap with at least a portion of thefirst vehicle image.
 5. The display control device according to claim 4,wherein the display processing unit displays the first vehicle image andthe second vehicle image in a display mode in which an overlappingportion between the first vehicle image and the second vehicle image isidentifiable.
 6. The display control device according to claim 5,wherein the display processing unit displays the first vehicle image andthe second vehicle image without executing a hidden surface removalprocessing on the overlapping portion between the first vehicle imageand the second vehicle image when displaying the first vehicle image andthe second vehicle image as 3D polygons.
 7. The display control deviceaccording to claim 1, wherein the display processing unit adjusts atleast one of a color and a brightness of the second vehicle imageaccording to at least one of a difference between the color of thesecond vehicle image and a color of a peripheral portion of theperipheral image around the second vehicle image and a differencebetween the brightness of the second vehicle image and a brightness ofthe peripheral portion.
 8. The display control device according to claim1, wherein the display processing unit generates the first vehicle imagebased on an overall shape of the vehicle including a wheel, andgenerates the second vehicle image based on a shape of a vehicle body ofthe vehicle including no wheel.
 9. The display control device accordingto claim 1, wherein the display processing unit changes the display modeof at least one of the first vehicle image and the second vehicle imageaccording to the remaining distance when the vehicle moves toward thetarget position under proxy control that executes at least a part of adriving operation of the vehicle by a driver.
 10. The display controldevice according to claim 9, wherein the target position is a parkingposition where the vehicle finally reaches in automatic parking controlas the proxy control or a quick-turn position where there is apossibility of the vehicle temporarily stopping before reaching theparking position.